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(54) Downloadable user-interface 

(57) Scheme and apparatus for controlling a com- 
puter device (10) with limited user-interface (11) via a 
remote computer device (12), whereby both computer 
devices are interconnected through a wireless commu- 
nication channel (16) and both computer devices sup- 
port a common communications protocol. In order to al- 
low control of the computer device (1 0) with limited user- 



interface (11) some user-interface information is sent 
from the computer device (10) with limited user-inter- 
face (11) to the remote computer device (12). At this re- 
mote computer device (12) a user-interface (1 9) is pro- 
vided. Then some user input is received at the remote 
computer device (12) and this user input is sent to the 
computer device (10) with limited user-interface (11) 
where the user input is executed. 
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Description 

TECHNICAL FIELD 

[0001] The invent jon concerns computer systems 
which have a limited user-interface, such as handheld 
pervasive computing devices, and in particular a 
scheme for allowing easy interaction with such compu- 
ter systems. 

BACKGROUND OF THE INVENTION 

[0002] Computer systems have become dramatically 
smaller and more portable. Even very powerful personal 
computers (PCs), for example, are small enough to sit 
on the desk at work. Smaller still are lap top computers 
and notebook computers. There are computer terminals 
which are small enough to be mounted in a vehicle such 
as a delivery truck. Still smaller are the hand held termi- 
nals typically used for their portability features where the 
user can carry the terminal in one hand and operate it 
with the other. 

[0003] There is a trend to offer consumer electronic 
devices that comprise some sort of a computer system, 
e.g., a microprocessor. Usually, these computer sys- 
tems not only control the operation or function of the 
consumer device, but also provide some interface for a 
user or operator to control certain functions or parame- 
ters according to actual needs. It is in the nature of these 
consumer devices that they do not have a full user-in- 
terface like a computer with display and keyboard. It is 
not likely, that a dishwasher, for example, will ever have 
such a full user-interface. In some cases the interface 
is limited because there are space constraints (a typical 
example for this is a wrist watch), in other cases the in- 
terface is limited to keep the cost of manufacturing low, 
or the processing power of the computer system or the 
constrained memory space limits the interaction be- 
tween the user and system. 

[0004] Many of the existing devices have an inade- 
quate user-interface. A typical examples is a compact 
disk (CD) player which allows programming of CD titles 
using a small four button-control. Programming of such 
a CD player is very cumbersome because one needs to 
use the buttons to move through the entire alphabet to 
select letters and/or numbers. Another example is a 
wrist watch that allows to enter phone book entries, ap- 
pointments, and to-do items. Usually, there is some sort 
of small keyboard with a very limited number of keys. 
Furthermore, the display is small and its resolution lim- 
ited. Certain keys have to be pressed several times to 
reach special characters, or to activate special func- 
tions. Yet another example is a personal digital assistant 
(PDA) with touch sensitive screen. In this case the 
screen occupies most of the device's surface and there 
are no or almost no buttons. Some functions are easily 
accessible using a pointing device, but other functions 
have to be selected or activated flipping through several 



layers of menus : for example. Other examples are tele- 
phones, vending machines, microwave ovens, mobile 
phones, etc. For the purposes of the present description 
these devices are referred to as user-interface limited 
5 devices. 

[0005] There exist a few approaches to use a person- 
al computer (PC) to run a better user-interfaces, e.g. the 
"Nokia Cellular Data Suite" for mobile phones that al- 
lows the entry of phonebook data, easier composition 

to of SMSes. The Cellular Data Suite is a hardware and 
software package from Nokia designed for cellular 
phones. Another example is a wrist-watch that has an 
IR-communication feature (such as the Casio PC Unite 
Data Bank Watch, HBX-100B-1) used to connect to a 

1$ PC. 

[0006] There are many other examples of user-inter- 
faces that are severely lacking for various reasons, the 
most prominent one certainly being size and cost con- 
straints. Often such user-interface restrictions make the 
20 respective devices less useful for their owners than they 
could be. 

[0007] There should be a way to unleash the full po- 
tential of all these devices and to program and configure 
them much more conveniently. 

25 [0008] There is growing demand to offer devices that 
are 'open' in the sense that a user has access via an 
interface to the device's processor or other components. 
An ideal 'open' device can be fully controlled by the user, 
preferably within well-defined rules to prevent misuse or 

30 destruction of the device itself. 

[0009] There is another unrelated trend. There is a 
growing number of devices that are network enabled 
which means that they can communicate with one or 
more other devices via a network. This can be achieved 

35 using physical connections, such as cables or fibers, for 
example. The smaller the devices get, the more impor- 
tant it becomes to replace these physical connections 
by wireless connections (e.g. body networks, radio fre- 
quency connections, or infrared connections), since 

40 physically connecting the devices by means of cables 
or fibers severely reduces the efficiency gained by mak- 
ing the units smaller Ad-hoc wireless connections are 
required where devices move around, enter an area and 
exit the area. The term ad-hoc refers to the need for fre- 

45 quent network reorganization. 

[0010] There are many different communications ap- 
proaches known that have been developed and de- 
signed with an eye on the communication between de- 
vices or subsystems. In the following some wireless 

50 communications approaches will be mentioned. There 
are many fiber or cable-based, standardized approach- 
es that are suited as well. 

[0011] GTE Corporation has developed a short-range 
radio-frequency (RF) technique which is aimed at giving 
55 mobile devices such as cellular phones, pagers and 
handheld personal computers (PCs) a smart way to in- 
teract with one another. GTE's technique is tentatively 
named Body LAN (local area network). The original de- 
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velopment of Body LAN was via a wired vest with which 
various devices were connected (hence the name Body 
LAN). This graduated to an RF connection a couple of 
years ago. 

[0012] Xerox Corporation has developed a handheld s 
computing device called PARC TAB. The PARC TAB is 
portable yet connected to the office workstation through 
base stations which have known locations. The PARC 
TAB base stations are placed around the building, and 
wired into a fixed wired network. The PARC TAB system io 
uses a preset knowledge of the building layout and the 
identifiers of the various base stations to decide where 
a PARC TAB portable device is by the strongest base 
station signal. A PARC TAB portable device has a wire- 
less interface to the base stations. The PARC TAB sys- 16 
tern assumes that the PARC TAB portable device is al- 
ways connected to the network infrastructure. The loca- 
tion of each portable PARC TAB device is always known 
to the system software. The base stations establish re- 
gions and are connected to power supplies. PARC TAB 20 
communication systems have a star topology. 
[001 3] In an attempt to standardize data communica- 
tion between disparate PC devices several companies, 
including Ericsson, IBM ; Intel, Nokia, and Toshiba es- 
tablished a consortium to create a global standard for 25 
wireless RF-based connectivity between fixed, portable 
and mobile devices. There are many other adopter com- 
panies. The proposed standard comprises an architec- 
ture and protocol specifications ranging from the phys- 
ical layer up to the application layer. The Bluetooth tech- 30 
nology will allow users to connect a wide range of de- 
vices easily and quickly, without the need for cables, ex- 
panding communications capabilities for mobile com- 
puters, mobile phones and other mobile devices. The 
Bluetooth operating environment is not yet fully defined, 35 
but there are expected to be similarities with the IrDA 
(Infrared Data Association) specification and the Ad- 
vanced Infrared (Air) specification. Other aspects that 
probably will find their way into Bluetooth might stem 
from the IEEE standard 802.11 and/or HIPERLAN, as *o 
promulgated by the European Telecommunications 
Standards Institute (ETSI). 

[0014] Bluetooth radio technology provides a mecha- 
nism to form small private ad-hoc groupings of connect- 
ed devices away from fixed network infrastructures. 45 
Bluetooth makes a distinction between a master unit - 
which is a device whose clock and hopping sequence 
are used to synchronize all other devices - and slave 
units in the same network segment. In other words, the 
Bluetooth approach is centralized. A query-based dis- 50 
covery scheme is used for finding Bluetooth devices 
with an unknown address. Queries are also centralized 
at a registry server. Further details can be found in 
Haartsen, Alien, Inouye, Joeressen, and Naghshineh, 
"Bluetooth: Vision, Goals, and Architecture" in the Mo- 55 
bile Computing and Communications Review, Vol. 1 , 
No. 2. Mobile Computing and Communications Review 
is a publication of the ACM SIGMOBILE; 



[0015] HomeRF (based on Shared Wireless Access 
Protocol (SWAP) is another example of an operating en- 
vironment which can be used to connect devices. A 
HomeRF Working Group was formed to provide the 
foundation for a broad range of interoperable consumer 
devices by establishing an open industry specification 
for wireless digital communication between PCs and 
consumer electronic devices anywhere in and around 
the home. The working group, which includes the lead- 
ing companies from the personal computer, consumer 
electronics, peripherals, communications, software, 
and semiconductor industries, is developing a specifi- 
cation for wireless communications in the home called 
the SWAP. The HomeRF SWAP system is designed to 
carry both voice and data traffic and to interoperate with 
the Public Switched Telephone Network (PSTN) and the 
Internet; it operates in the 2400 MHz band and uses a 
digital frequency hopping spread spectrum radio. The 
SWAP technology was derived from extensions of ex- 
isting cordless telephone (DECT) and wireless LAN 
technology to enable a new class of home cordless serv- 
ices. It supports both a time division multiple access 
(TDMA) service to provide delivery of interactive voice 
and other time-critical services, and a carrier sense mul- 
tiple access/collision avoidance (CSMA/CA) service for 
delivery of high speed packet data. The SWAP system 
can operate either as an ad-hoc network or as a man- 
aged network under the control of a connection point. In 
an ad-hoc network, where only data communication is 
supported, all stations are equal and control of the net- 
work is distributed between stations. For time critical 
communications such as interactive voice, the connec- 
tion point - which provides the gateway to the PSTN - is 
required to coordinate the system. Stations use the CS- 
MA/CA to communicate with a connection point and oth- 
er stations. Further details about HomeRF can be found 
at the Home Radio Frequency Working Group's website 
http://www.homerf.org. The SWAP specification 1.0 is 
incorporated by reference in its entirety. 
[0016] There are several more or less elaborate pro- 
tocols and techniques that allow a communication be- 
tween two or more devices. The above described Blue- 
tooth radio technology and HomeRF approach are 
prominent wireless examples. 

[0017] It is an object of the present invention to pro- 
vide a scheme for providing a more powerful user-inter- 
face to an interface limited device. 
[0018] It is another object of the present invention to 
provide a scheme for simplified and/or improved human 
interaction with an interface limited device. 

SUMMARY OF THE INVENTION 

[0019] The present invention concerns a scheme, as 
claimed, for controlling a computer device with limited 
user-interface using a computer device with a more 
powerful and/or better user interface. 
[0020] The present invention concerns a system com- 
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prising a computer device with limited user-interface 
and a computer device with a more powerful and/or bet- 
ter user interface, as claimed. 

[0021] The present invention concerns a computer 
program product, as claimed. 

[0022] With the present invention, a solution is pre- 
sented that, when used in combination with a suited 
communications protocol, allows a user to interact with 
or control an interface limited device using a second (in- 
dependent) device. 

[0023] In other words, the present invention provides 
an improved user- interface to a user-interface restricted 
device by using a more powerful device in its vicinity. 
The better input capabilities of the more powerful device 
are employed to control certain aspects of the user-in- 
terface restricted device. The present invention allows 
to provide a more intuitive user-interface, for example. 
Devices according to the present invention allow a sim- 
plified and/or improved human interaction with an inter- 
face limited device. 

[0024] The present scheme facilitates various imple- 
mentations. 

DESCRIPTION OF THE DRAWINGS 

[0025] The invention is described in detail below with 
reference to the following schematic drawings. It is to 
be noted that the Figures are not drawn to scale. 

FIG. 1 is a schematic block diagram of an embodi- 
ment, in accordance with the present inven- 
tion. 

FIG 2. is a schematic representation of an exem- 
plary execution tree. 

FIG. 3. shows a first browser window. 

FIG. 4. shows a second browser window. 

FIG. 5. shows a third browser window. 

FIG. 6 is a schematic block diagram of another em- 
bodiment, in accordance with the present in- 
vention. 

FIG. 7 A is a schematic block diagram of an embodi- 
ment, in accordance with the present inven- 
tion. 

FIG. 7B is a schematic block diagram of the logical/ 
software elements of an embodiment, in ac- 
cordance with the present invention. 

FIG. 7C is a schematic block diagram of the hard- 
ware layer of an embodiment, in accordance 
with the present invention. 



FIG. 7D is a schematic block diagram of the hard- 
ware layer of an embodiment, in accordance 
with the present invention. 

5 FIG. 8 is a schematic flowchart used to describe as- 
pects of a user-interface limited device : in 
accordance with the present invention. 

FIG. 9 is schematic flowchart used to describe as- 
1( > pects of a device that is used to control a us- 

er-interface limited device, in accordance 
with the present invention. 

FIG. 10 A Wireless Markup Language document de- 
15 scribing the user-interfaces of Figures 3-5. 

DESCRIPTION OF PREFERRED EMBODIMENTS: 

[0026] For the purpose of the present description, a 

20 network can be anything that allows a first device (the 
user-interface limited device) to communicate with a 
second device (which has a better user-interface, for ex- 
ample). A simple point-to-point link, a local area network 
(LAN), a GSM telephone link, an ethernet link or any 

25 other kind of link is herein for sake of simplicity referred 
to as network. This network can either be a physical net- 
work or a wireless network (e.g., infrared (IR), radio-fre- 
quency (RF), such as HomeRF). The network may be 
completely isolated from any other network, or it might 

30 comprise one or more access points which provide the 
devices with access to the another network. 
[0027] The specific range that constitutes a wireless 
network in accordance with the present invention de- 
pends on actual implementation details. Generally, a 

35 wireless network can be described as having a cover- 
age area between a few square meters and several 
thousands of square kilometers (e.g., in case of a GSM 
network). Under certain circumstances the communica- 
tion range might even go beyond. The two devices 

40 which communicate with each other have to be in vicinity 
which means that they have to be able to exchange in- 
formation with each other. 

[0028] The devices need to be able to transmit and/ 
or receive information via the network. For this purpose 

45 two devices that communicate with each other have to 
support the same communication protocol. 
[0029] Well suited for this communication between 
devices is the Bluetooth communications scheme, 
which is described in the Haartsen, Allen, Inouye, Joer- 

50 essen, and Naghshineh, "Bluetooth: Vision, Goals, and 
Architecture" in the Mobile Computing and Communica- 
tions Review, Vol. 1 , No. 2. Mobile Computing and Com- 
munications Review is a publication of the ACM SIG- 
MOBILE. This reference is incorporated by reference in 

55 jts entirety. 

[0030] It is assumed, that once the devices are in vin- 
cinity of each other a wireless communication path be- 
tween these devices can be established - e.g. using 
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magnetic field (near field / 5 - 30 cm), infrared (IR) : e.g., 
IrDA (0.5 - 2m) or Air (1 -10m), or low-power radio-fre- 
quency (RF) communication, e.g. BlueTooth (~1 - 10m), 
or HomeRF (~1 - 50m), just to give some examples of 
wireless schemes that are suited. s 
[0031] Of course this vicinity requirement could also 
be loosened given some global addressing scheme and 
a local proxy (e.g. an IR-beacon on each ceiling or a 
BlueTooth "relay" in each room or each flat) connected 
to some network (e.g. the Internet) thus effectively al- 10 
lowing remote control of ail kind of devices from virtually 
anywhere on the globe. Such a global addressing 
scheme is also required in a GSM-based implementa- 
tion of the present invention. 

[0032] It is understood by those skilled in the art that is 
at the present time many of the protocols that are suited 
for use in wireless communications systems are still in 
draft status. The present scheme is independent of any 
particular protocol and can be used in connection with 
many such protocols. Somebody skilled in the art is able 20 
to implement the present scheme in existing protocol 
environments as well as in protocol environments under 
development or yet to be developed. 
[0033] The present scheme can be used anywhere in- 
side (warehouses, on manufacturing floors, in offices, 25 
on trading floors, in private homes) and outside of build- 
ings, in cars and trucks, in airplanes, just to mention 
some examples. 

[0034] Two devices can be connected using a 1:1 
connection. Possible media are infrared and magnetic 30 
field. The procedure to setup such a 1 :1 connection can 
be similar to today's setup of a connection between two 
IrDA enabled devices, i.e. the devices must be posi- 
tioned such that their communication subsystems 
(transceivers) can "see" each other. Then both systems 35 
are triggered to start a connection setup procedure until 
a wireless communication channel is established. 
[0035] Likewise, two devices can be connected using 
a shared medium. A possible shared medium is obvi- 
ously RF (Radio Frequency). Possible systems could be 40 
based on technology and protocols like BlueTooth, 
DECT, and HummingBird. 

[0036] Details about HummingBird transceivers are 
given in "Hummingbird Spread Spectrum Transceiver 
Operator's Manual", Rev 24 June, 1998, XETRON 45 
Corp., Cincinnati, Ohio, USA 

[0037] Details concerning the basic problems re. 
identification and addressing, initial (resource) discov- 
ery; matching and selection of communication partners, 
etc. depend on the medium used and the communica- so 
tions protocol employed. 

[0038] When referring to a device, any kind of device 
is meant that can establish a network connection to an- 
other device. Examples of devices are: laptop comput- 
ers, workpads, node pads, personal digital assistants S5 
(PDAs), notebook computers and other wearable com- 
puters, desktop computers, computer terminals, net- 
worked computers, internet terminals and other comput- 



ing systems, set-top boxes, cash registers, bar code 
scanners, point of sales terminals, kiosk systems, cel- 
lular phones, pagers, wrist watches, digital watches, 
badges, and smart cards. Other contemplated devices 
include: headsets. Human Interface Device (HID) com- 
pliant peripherals, data and voice access points, cam- 
eras, printers, fax machines, keyboards, joysticks, HiFi 
systems, audio (sound) cards, loudspeakers, amplifiers, 
video cards, kitchen appliances, tools, sensors such as 
smoke and/or fire detectors, and virtually any other dig- 
ital device. 

[0039] Other examples of devices that can be used in 
connection with the present invention are, personal ef- 
fects being equipped with computer-like hardware, such 
as a "smart wallet" computer, jewelry, or articles of cloth- 
ing, in addition to a "smart wallet" computer, there are 
a number of other variations of these so-called wearable 
computers. A "belt" computer is such a variation which 
allows the user to surf, dictate, and edit documents while 
they are moving around. Yet another example is a child's 
computer which is comparable to a personal digital as- 
sistant for grade-school children. The child's computer 
might hold assignments, perform calculations, and help 
kids manage their homework. It can interface with other 
children's computers to facilitate collaboration, and it 
can access a teacher's computer to download assign- 
ments or feedback. Any wearable or portable device, 
any office tool or equipment, home tool or equipment, 
system for use in vehicles, or systems for use in the pub- 
lic (vending machines, ticketing machines, automated 
teller machines, etc.) might comprise the present inven- 
tion. 

[0040] It is furthermore assumed that a device, as 
used in connection with the present invention, has a 
minimum amount of processing power that enables it to 
participate in the scheme according to the present in- 
vention. These devices are thus also referred to as com- 
puter devices. Most, if not any, of the above listed de- 
vices can be viewed as being devices with limited user- 
interface. This is even the case for a personal computer 
which has a display and a keyboard. There still is room 
for improvement of such a computer's interface, e.g., by 
adding speech input. There are no absolute criteria 
which can be used to decide whether a particular device 
is a device with limited user-interface or not. There is 
always room for improvement and thus any computer 
device per se is assumed to be a device with limited us- 
er-interface. There present scheme works in any con- 
stellation where there is a second computer device that 
has a more powerful user-interface, more adequate, 
more convenient, or better user-interface capabilities. 
Not all aspects of the user-interface have to be better or 
more powerful, it is sufficient, for example, if there is a 
first devices which has no speech input (i.e. it has a lim- 
ited user-interface) and a second device which has a 
speech input. 

[0041] Some of the above-mentioned devices can be 
regarded as the device (controller) whose interface is 
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used to interact with the user-interface limited device 
(controlled device). 

[0042] A computer device is referred to as a computer 
device with limited user-interface if one or more of the 
following applies: 5 

• it has an inadequate user-interface; 

• it has a user-interface that is small, difficult to read, 
understand, or hear; 10 

• it has a user-interface that is not convenient: 

• it has no graphics capable display (e.g. a text-only 
display); is 

• it has a restricted number of input keys, or input 
keys which are small; 

• it has many functions which are mapped to a limited 20 
number of buttons and thus imposes complicated 
control structu res that make it difficu It to operate the 
device without prior extensive study of a user's 
manual (especially for seldom used or advanced 
functions); 2$ 

• it has a user-interface which is not powerful enough 
such that it istooslow, has low resolution, or the like. 

[0043] Devices with better user-interface capabilities 30 
usually fulfill one or more of the following criteria: 

• it has a large screen: 

• it has a screen with graphics capability; ss 

• it has a full keyboard; 

• it has a pointing device; 

40 

• it has voice-input, and so forth. 

[0044] Please note that the user-interface can be any 
kind of interface for interaction between a user and the 
device, such as a display, keyboard, mouse, track point, 45 
audio output, speech recognition input, tactile input, etc. 
[0045] A typical environment where the present in- 
vention is used is illustrated in Figure 1. There is a first 
computer device 10 which has a limited user-interface 
1 1 (in the present example the user-interface comprises so 
a simple display and a few buttons). Assuming, that 
there is a second computer device 12 "nearby" (i.e., in 
vicinity of the first device 10) that has better user-inter- 
face capabilities (keyboard 13 and display 14) than the 
first device 10, that we want to control or configure and ss 
assuming, that the two devices 10 and 12 find a way to 
communicate with each other, this allows the deploy- 
ment of the better user-interface capabilities of said sec- 



ond device 12 to ease and speed-up the use of the first 
device 10 by shipping a better user-interface (user-in- 
terface description 1 5) to the second device 1 2. The first 
device 10 and second device 12 communicate through 
a wireless communications channel 16. A better user- 
interface is a user-interface that is easier to use, "richer" 
(in that it has more features or contains explanations, 
for example), more intuitive, faster, or the.like. The user- 
interface description 1 5 is then processed by the second 
device 12 such that a better user-interface 19 is then 
displayed and operated on the second device 12. Then 
user-inputs and/or commands and/or parameters are 
sent back to the first device for execution. In the present 
example commands 17 (<command>) and parameters 
18 (<parameters>) are sent back to control or operate 
the device 1 0. 

[0046] The capability that a first device 1 0 can provide 
its user-interface in some standard format (herein re- 
ferred to as user-interface description 15) may be broad- 
cast to all other devices (including the above-mentioned 
second device 12) appearing in its vicinity. If the user- 
interface description 15 is sufficiently small then the en- 
tire interface description can be sent right away and 
stored at these other devices. 

[0047] If there are a plurality of devices ("controllable" 
devices) with limited user-interface within vicinity of a 
second device with better user-interface, then the user 
can request some visualization on the second device of 
all "controllable" devices (e.g., in the form of a list, a 
menu, a graph, or the like) from which he/she can then 
choose one device with limited user-interface and re- 
quest its user-interface to be displayed to start the proc- 
ess, according to the present invention. 
[0048] The communication path 16 between the first 
device 1 0 and second device 1 2 is used to transfer data 
describing a suitable user-interface (user-interface de- 
scription 15) for a specific transaction from the first de- 
vice 10 (thus becoming the "controlled device" or server) 
to the "nearby" second device 12 (the controller or client/ 
user agent). The second device 12 renders the user- 
interface description 15 to the user. This can be done 
by displaying it (reference number 19) to the user on 
display 14, for example. Then, the second device 12 
awaits the user's reaction. 

[0049] The user answers the posed question (s), e.g. 
by picking his/her choice from a presented menu, or 
supplies input by keying-in the requested data. In doing 
so the user can make use of the controller-device's bet- 
ter user-interface capabilities (be it a larger keyboard 
13, voice-recognition, color-display 14, or the like). In- 
formation describing the user's reaction, selection, or in- 
put is then sent back to the controlled device in the form 
of "requests" (i.e. commands 17 and - optionally - one 
or more parameters 18). 

[0050] The major point of the inventive approach is, 
that the controlling device 1 2 does not need to have any 
prior knowledge of the features and the user-interface 
of the controlled device 10. No special software needs 
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to be pre-installed because everything is dynamically 
downloaded when required. Any Laptop or PDA that 
happens to be handy or maybe even a public kiosk sys- 
tem could quickly be used as "user-interface server" 
without need to install anything nor without leaving any s 
traces on that system (except maybe a few modified 
cache-entries in the system's memory). However, there 
are of course some prerequisites that have to be fulfilled 
by all involved devices to allow such a scenario to be- 
come reality. io 
[0051] Standardized user-interface description: 
There must be a standardized way and format to de- 
scribe sufficiently rich user-interfaces such that it is pos- 
sible to render typical user-interface controls, i.e. dis- 
play input-prompts, selection-menus, help-texts or other 1$ 
text-messages to visualize a device's status, etc. Suit- 
able candidates for such user-interface description for- 
mats are: 

• HTML (the HyperText Markup Language used in the ?o 
WWW), 

• WML (Wireless Markup Language defined by the 
WAP forum), 

25 

♦ other, still-to-be-defined XML (Extensible Markup 
Language) dialects, 

♦ X-windows protocol. 

30 

[0052] One preferably uses a user-interface descrip- 
tion which is optimized so that transmissions between 
devices are efficient. The user-interface description 
should be flexible and extensible. 

[0053] Standardized Communication: The devices 35 
must be able to detect each other's presence, exchange 
basic capability description and - on demand - be able 
to setup a sufficiently reliable point-to-point connection 
amongst each other. This basic capability description 
can be a simple flag or bit combination, for example, 40 
which describes standard types of services. These 
standard type of services might be predefined such that 
they can be identified by such a simple flag or bit com- 
bination. The basic capability description can also be 
any other kind of information which is suited to identify 45 
one or several services offered. In addition to identifying 
a type of service, one might have to set or define certain 
parameters and options (for sake of simplicity hereinaf- 
ter referred to as parameters). 

[0054] There are thus some common requirements, so 
To this end it is assumed, that the devices run some kind 
of resource discovery scheme and exchange some form 
of capability and/or device-class description once they 
detect their mutual presence. A device must be able to 
become aware of its neighborhood, to discover potential 55 
communications peers within mutual reach, and their 
service offerings. In addition, it is advantageous if a de- 
vice is able to indicate its presence and to advertize its 



own service, if any. The advertisement of service infor- 
mation can be done in another protocol layer. Another 
approach is conceivable where it is inherently known 
which service is offered by what device (i.e., all devices 
XYZ offer the services A and B, and all devices MNO 
offer the services C and D, for example). 
[0055] An example of a scheme for advertisement 
and/or discovery of services is addressed in a co-pend- 
ing European patent application entitled "Service Adver- 
tisements in Wireless Local Networks", filed on 25 Jan- 
uary 1999, currently assigned to the assignee of the 
present application. According to this scheme, each de- 
vice will take turns broadcasting (advertising) a list of 
services (among which could be the ability to send user- 
interface description and receive the corresponding 
commands) available. The general approach is that a 
group of devices will take turns broadcasting (advertis- 
ing) a list of services (herein referredto as user-interface 
description) available. By using variable transmission 
delays that are reset when other advertisements are 
seen, and adjusting the distribution of these delays, new 
devices can quickly be identified, and absent machines 
can be noticed. This scheme allows to form small private 
ad-hoc groupings of connected devices. The scheme al- 
lows to set up local networks immediately (ad-hoc) if 
needed, and to take them down if not needed anymore. 
According to this scheme a network of all eligible prox- 
imate devices (devices that will allow themselves to be 
networked) can be set up while allowing new devices to 
join and leave at their own convenience. 
[0056] The present invention is independent of the 
scheme for advertisement and/or discovery of services. 
What is required is that a service-consuming device (j. 
e. the device which has a better user-interface) knows 
or learns about service-providing devices (i.e. those de- 
vices with limited user-interface) within vicinity. For this 
purpose the service-consuming device stores service 
information (e.g. a list of entries describing the capability 
to provide a user-interface description of other devices) 
identifying services of which it is aware of. The service 
information has to be updated frequently, since a wire- 
less network might change from time to time. 
[0057] One example of a resource discovery scheme 
is described in the following. This scheme allows two 
devices within vicinity to determine whether certain 
services are available and what kind of services there 
are. One device maintains a record with information 
about services and associated identifiers about the oth- 
er device which acts as service-providing device. The 
one device may comprise a service discovery module 
which maintains a record with information about servic- 
es and associated identifiers, and a list of identifiers 
about service-providing devices. The service discovery 
module enables the device to distinguish a service of- 
fered by a service-providing device in vicinity from a 
service offered by a service-providing device not being 
in vicinity. The resource discovery scheme can be de- 
fined such that it, when used in combination with a wire- 
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less communications protocol, allows to ensure/control 
that certain services or tasks are carried out or assigned 
to a device with better user-interface which is within ad- 
jacency of the device (the device with limited user-inter- 
face) which requests the service. 
[0058] The device's capability description may in- 
clude a basic initial message (a URL / a "pointer" - at 
most 100.. 200 bytes - see later for more details) which 
is stored at the receiver side as part of each device's 
description, for example. 

[0059] The controlled device must be able to send us- 
er-interface descriptions in some standardized format 
and to receive and interpret inputs, commands and/or 
parameters sent back from the controlling device. 
[0060] The controlling device must understand and be 
able to receive these user-interface description and to 
make them accessible to the user (on demand or auto- 
matically) - e.g. by displaying a menu or a list of all de- 
vices currently in reach. The controlling device further- 
more must be able to forward commands to the control- 
led device. 

[0061] The basic idea of the present invention will be 
described in connection with an embodiment. In the fol- 
lowing, the invention is implemented and described in a 
communications system using WML The benefits of 
WML are: 

• small, simple, XML-based "language". 

• relatively small and simple browser, that will soon 
be available on many mobile devices. 

• "deck-of-cards" metaphor maps perfectly to familiar 
configuration paradigms ("property-sheets" or "- 
tabs"). 

• WML can be encoded very compact (binary WML 
or tokenized WML), i.e. most tags become single- 
byte items and strings are collected in a string-table. 
In the example given below the original WML file is 
1.5kB, while the tokenized form is 652 bytes only 
and that is even without compressing the string ta- 
ble, yet. If that table would be compressed (e.g., us- 
ing Lempel-Ziv, the same algorithm used in the 
popular .zip or .gzip tools) the resulting files were 
even smaller. 

• has built-in variables replacement functionality. 

• has timer functionality 

[0062] Prerequisites: To signal that certain com- 
mands (and parameters) are to be sent to the controlled 
device the WML-browser (or its underlying communica- 
tion stack) must recognize URLs that use a special 
"scheme" or "protocol". 
[0063] Based on existing schemes, like: 



• http://<host>/<path>[; <params>][?<query>][#<an- 
chor>] for HTTP (HyperText Transport Protocol, i.e. 
the Internet-WWW protocol) requests as specified 
in [RFC 1738] and [RFC2068]; 

s ftp://<host>/<path>/<filename> 

for the file transfer protocol; and 

gopher://<host>/<path>/<fiiename> 

for the. gopher protocol, there have already been 

suggestions to extend this notation by additional 
io schemes. 

• One that has already widely been adopted and in- 
corporated into most WWW-browsers is the "file: u - 
scheme: 

file://<host_name>/<local_path>/<filename> 
15 (Note: the //<host_name> fragment is optional and 
- if not present - defaults to "this host" or "local host" 
which accesses a local file rather than a file or re- 
source located at some server). 

• Similarly, but not yet adopted, there has been the 
20 suggestion to add a scheme that allows the control 

of and communication via "local" devices (serial 
ports, printer ports, smart-card readers, USB-ports, 
etc.) using: 

device://<portname>/<cmd>[;<params>][?<que- 
25 ry>][#<anchor>] 

(Note [...] indicates optional parts) - e.g. device: // 
COMI/setbaudrate; 1 9200 to change the first com- 
munication port's serial speed to 1 9200 baud/s. 

• We propose (and implemented) a similar scheme 
30 to send commands to devices that are "attached" 

using some short-range communication means 
(like IR or RF) using 

<comm>://<device_id>[:<portnr>]/<prefix>/<cmd> 
[;<params>.] 

35 Under <comm> we understand the network or com- 
munication means over which this command/re- 
quest is to be sent, e.g. "irda" or "bluetooth", "hum- 
mingbird", etc. 

<device_id> is needed for communication means, 
40 that support multi-party communication (i. e. not on- 
ly 1:1 communication like IrDA V1) to address a 
specific device. The optional part :<portnr> can be 
further used to specify a specific port, in case more 
than one communication channel exists between 
45 these devices or to choose a non-default channel 
for special purposes (e.g. for device monitoring, di- 
agnosis, configuration, etc.). 
This device-id could be the concatenation of some 
manufacturer- and model-id (e.g. 
50 "sony_cdp_990X") with some user-specified arbi- 
trary name or physical location (e.g. 
"mmoserjivingroom"') 

<prefix> (which has the same syntax like a URL- 
<path>) can be used to group commands into some 
55 tree-structure, e.g. if one looks at a printer's menu- 
tree ^execution tree) as shown in Figure 2. 
<cmd> (or the last path-fragment) specifies the ac- 
tual command, while <params> describes optional 
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parameters of the above command: 
Examples: 

bluetooth://sony_cdp_990_mmoserjivingroom/ 
cd_tities/add; BMW:Exodus 
bluetooth://sony_cdp_990_mmoserjivingroom/ 
play_mode/select; shuffle. 

Resource discovery: The device sends a string in the 
format 

wmLui=<device_id>/[<path>]/<command>[;<parame- 
ters>] 

for example: 

wml_ui=sony_cdp_990_mmoser_livin groom/ 
main_menu 

when describing its resources to its peers. In the above 
example wml_ui is a predefined service name (stand- 
ardized) and 

sony_cdp_990_mmoserjivingroom/main_menu is an 
initial URL. That URL decomposes into a "host name" 
sony_cdp_990„mmoserjivingroom which is a logical 
name that has to be mapped to the physical address of 
the device and the remainder (command and optional 
parameters) which is the actual request to be sent to the 
device, in this example the command "main_menu" 
(without parameters) to return the initial main menu. 
[0064] This URL is stored as part of the description 
that all devices maintain about other devices currently 
in reach. If the user-interface is small enough the device 
might even send the entire user-interface description 
right away. 

[0065] Selection of service: Devices, that can act as 
service-providers and controllers (e.g. PDAs, Laptops, 
PCs, ...) for a device with limited user-interface have 
some means to display the user-interface description re- 
ceived as user-interface. If there are a plurality of devic- 
es with limited user-interface in vicinity of a device that 
serves as controller, a choice of this plurality of devices 
together with a user-interface capability-description can 
be displayed on the controller's screen. E.g., such a de- 
vice could include an "act as user-interface for a nearby 
device"-button in their "system-menu". Clicking on that 
entry would pop up the above mentioned list of "control- 
lable" devices. The user can pick one from that list 
whereupon the user-interface-URL (wml-user-inter- 
face-URL) is submitted to the selected device thus ini- 
tiating the process according to the present invention. 
[0066] The submission of the user-interface-URL trig- 
gers the transfer of the main-control menu of that device. 
WML uses a"deck of cards" metaphor which quite nicely 
maps to "property sheets", a visualization technique that 
is often used to edit object attributes and parameters. 
For the present CD-player example such a deck could 
look as shown in Figures 3 - 5 (for space and complexity 
reasons the example contains a deck with only 4 cards: 
a welcome & overview card, two cards to edit CD names 
and select the play mode and a generic help card). The 
corresponding WML document is given in Figure 10. 
[0067] If a WML-browser is used by the controller, the 



above deck would be displayed as window 30 on the 
screen 31 of the controller, as shown in Figure 3. If the 
user now clicks on the CD-labels link 32 or if the user 
selects the "CD_names"-tab 33, a card 40 to edit CD- 
5 titles is brought up. This card 40 is shown in Figure 4. 
Here the user can now enter a CD-name in an insert 
field 41 using the input facility of the controller device, 
e.g., a full-blown keyboard, pen-input, voice-input, or 
the like. 

io [0068] Selecting the play-mode link 35 (or 
play_mode-tab 34) pops up a window 50 as shown in 
Figure 5. Again, the user can choose the different piay 
modes by clicking on one of the radio buttons 51 using 
the controller device's pointing medium. 

15 [0069] Transfer of commands and parameters: When 
the user clicks an OK-button, e.g., the OK-button 52 on 
the CD-player play-modes window 50 of Figure 5, the 
controller's browser submits e.g. the following URL: 
bluetooth://sony_cdp_990/playmode/select;Normal to 

20 switch the play mode to "Normal". Based on the scheme 
or protocol (here "bluetooth") the controller device's 
communication stack recognizes, that this is not a nor- 
mal request to sent out via TCP/IP and the Internet but 
rather that this must be intercepted and forwarded to the 

25 local (bluetooth) communication stack. 

[0070] The host specification of the URL is then used 
to address the specified device (here a "Sony CD-player 
model 990") and the remaining URL-part (the optional 
path, the command and the optional parameter(s)) are 

30 then sent to the specified device. 

[0071] Recognition and execution of commands: The 
addressed device must have a simple "command-inter- 
preter" that is able to analyze submitted URLs, i.e. ex- 
tracts and recognizes certain command strings plus op- 

35 tionally separates and converts parameters, etc. The 
complexity and robustness of this interface is complete- 
ly up to the manufacturers discretion. 
[0072] Feedback: The user expects some reaction 
when he/she presses a button or clicks on a link and 

40 thus "submits a request" to the controlled device. For 
this reason that device may react (this is an optional 
step) and return some response to the submitted re- 
quest (just getting a time-out message in the browser 
and no success/failure indication whatsoever is usually 

45 not sufficient). 

[0073] The flexibility, size, and complexity of this re- 
sponse is completely up to the manufacturers discre- 
tion, the devices capabilities, and resources. The device 
could e.g. either 

50 

• return a specific card confirming the reception of the 
command and describingthe results of its execution 
(if any), or it 

55 • returns its complete user-interface-"deck" again 
(possibly with certain texts or default choices now 
adapted according to the status changes caused by 
the previous command, or it 
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♦ returns just a minima! OK- or error-page depending 
on the command's result and the user than has nav- 
igate back to the control stack by pressing "return" 
in the browser. 

[0074] Other reactions or combinations of the above 
are of course also possible. 

[0075] In the following an exemplary implementation 
of the present scheme is described in connection which 
Figure 7A. In this Figure a schematic block diagram of 
the components (note that some are logical components 
and others are physical components) of a device 70 - in 
which the present invention is implemented - is shown. 
The device 70 comprises a transmitter driver 73 for 
sending information via an output channel 81 to another 
device (such as a device with better user-interface ca- 
pabilities), and a receiver driver 74 for receiving through 
an input channel 82 information from another device. 
Note that in the present example there are two channels 
81 , 82 shown. These channels can be any kind of chan- 
nels, such as an IR, RF, or body network channels, for 
example. These channels do not have to be the same. 
It is conceivable that the output channel 81 is an infrared 
channel whereas the input channel 82 is a RF channel. 
[0076] The transmitter driver 73 and receiver driver 74 
communicate with a medium access control (MAC) unit 
72. The MAC layer is well defined by international stand- 
ards (cf. ISO OSI (Open Standards Interconnection) ref- 
erence model as described in A.S. Tannenbaum's book 
"Computer Networks"., for example) and the MAC unit 
72 might be a conventional unit employed in communi- 
cation systems to control the MAC layer. Note that a 
MAC layer is a logical division, and would be only logi- 
cally divided from other parts of the protocol implement- 
ed at 71 on the same physical device. The MAC unit 72 
might be employed to detect and/or avoid collisions. In 
the present embodiment the MAC unit 72 is used to send 
and receive packets. In many cases such a MAC unit 
72 is not required. 

[0077] The power might be provided by via a power 
plug, a solar cell, a battery, or the like. The power supply 
(not shown) provides power to the components of the 
device 70. For sake of simplicity, the respective circuit 
lines or cables are not shown in Figure 7. 
[0078] As shown in Figure 7D the device 70 may com- 
prise a bus 21 enabling communication between some 
of the device's components/units (such as the central 
processing unit (CPU) 77, memory 76, the communica- 
tion hardware 22, 23 and any other device specific hard- 
ware 20 through a hardware interface 25). The device 
70 may also have a user-interface unit 24 for interaction 
with a user (e.g. a small LCD display and some input 
keys). Note that the actual user interfaces are not shown 
in Figure 7A. 

[0079] For remote access user-interface information 
is fed from a user-interface manager 71 to the MAC unit 
72 and forwarded to the (remote) controlling device. Us- 
er-interface information refers to information needed by 



the device with better user-interface capabilities to pro- 
vide a user-interface to a user. Depending on the imple- 
mentation, user-interface information refers to informa- 
tion that describes a full user-interface (see item 19 in 
5 Figure 1 ), or it describes an initial URL or a partial user- 
interface (item 63 in Figure 6) - see below for a descrip- 
tion of the approach to provide a partial user interface 
only. 

[0080] On the backward path commands entered by 
10 the user are fed via MAC 72 and user interface manager 
71 to soft- and/or hardware controlling the device 70. To 
do so the user-interface manager 71 may communicate 
directly (item 83) or indirectly via an optional application 
programming interface 79 (API) and a device specific 
is application 78 with hardware drivers 26 and conse- 
quently with device specific hardware 20 that provides 
and embodies a device's purpose and/or functionality 
(be it a video cassette recorder, a coffee machine, a 
printer, a stereo device, etc.). The actual activity or func- 
tionality of the device 70 is independent of the present 
scheme. The important point is, that - using the present- 
ed scheme - this activity can be controlled and/or mon- 
itored from another device. 

[0081] Note that the MAC 72, the user-interface man- 
ager 71 and the application 78 are logical constructs. 
They could be implemented on separate devices, but 
they can equally well be incorporated into a program 
stored in memory 76. If incorporated into a program the 
device 70 might physically be the same as any other 
conventional device, except for the fact that it comprises 
the above-mentioned program. This program comprises 
instructions that, if processed by the CPU 77, make the 
device 70 perform the steps according to the present 
invention. 

[0082] The user-interface manager 71 implements at 
least part of the present scheme for exchange of user- 
interface information, allowing the user-interface to be 
provided at a remote device and control information and/ 
or parameters to be received from the device in re- 
sponse to user input. 

[0083] A schematic flow chart is given in Figure 8. This 
flow chart is used to describe the steps that are per- 
formed by a computer device with limited user-interface. 
In the present example this device is listening for infor- 
mation (box 87). From time to time the device may send 
some service information (service announcement) to 
one or more devices within vicinity (box 84). This listen- 
ing mode is optional. This service announcement proc- 
ess may run in the background, as indicated by the 
dashed loop on the left hand side of Figure 8. If the de- 
vice receives input (box 88) from another device (the 
second device), it categorizes this input into one of sev- 
eral categories. In the present example there are three 
categories: service input; request to send user interface 
(Ul) information; user input. If the input contains service 
information, then this information is used to update the 
device's own list of services (box 89). This service infor- 
mation may be used by the second device to transmit 
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information about its capabilities. This service informa- 
tion may be kept in a list so that it is always present if 
needed. Other schemes are conceivable where the in- 
formation is only fetched if needed, for example. If the 
input is identified to be a request to send user interface s 
(Ul) information then the device sends its Ui information 
to the second device (box 85). If the input comprises a 
user input, then the device handles and/or executes this 
user input (box 86). in an optional step (box 90) some 
feedback is returned to the second device to indicate to 10 
the user that the controlled device handled or executed 
his command. 

[0084] Alternatively (see the dashed arrow at the low- 
er righthand side) the device could send an entire or par- 
tial Ul description again, updated such, that it reflects 15 
results or status changes caused by the prior command. 
Finally, the device goes back into listening mode (box 
87). 

[0085] The embodiment described in connection with 
Figure 8 implements a scheme where the second device 20 
(i.e., the device with better or more powerful user inter- 
face) triggers the computer device with limited user-in- 
terface to send user interface information. This can for 
example be initiated in that a user points with the second 
device into the direction of the computer device with lim- 25 
ited user-interface. 

[0086] The embodiment of a controller 700 - i.e. a de- 
vice that has a better user-interface - is illustrated in Fig- 
ures 7B and 7C. Figure 7B shows some of the logical 
and software- layer building blocks and Figure 7C shows 30 
some building blocks of the hardware layer. As shown 
in Figure 7B, the device 700 comprises a MAC protocol 
handler 720, a transmitter driver 730 : and a receiver 
driver 740 for communication with a remote device (not 
shown). In addition there is a user interface manager 35 
710 and some driver 750 to communicate with the user 
interfaces. Again, the device 700 can comprise a bus 
706 (e.g. a back plane bus or a cable bus) for intercon- 
necting a transmitter 701, receiver 702, memory 703, 
CPU 704, and user interface 705 connected to a display 40 
and/or keyboard, pointing device, for example. 
[0087] The corresponding steps that are performed 
by the second device are illustrated in Figure 9. This 
second device may receive service information from the 
computer device with limited user-interface (box 103) if 45 
this device is set up to send such information from time 
to time. If there is a plurality of computer devices with 
limited user- interfaces (controllable devices) within 
reach of the second device, then - according to the 
present implementation example - a list of these con- so 
trollable devices is presented to the user (boxes 91 , 92). 
Then the user selects the device (controlled device) he 
wants to control or interact with (box 93). If there is only 
one controllable device or if the second device other- 
wise knows which controllable device the user wants to 55 
interact with, then the boxes 91 -93 can be bypassed as 
indicated by arrow 94. Now a request to send user in- 
terface information is sent to the controlled device (box 



95) . The controlled device responds by sending the re- 
quested user interface information. After this user inter- 
face information is received by the second device (box 

96) , a user interface is rendered to the user (box 97). 
This can for example be done by displaying the user in- 
terface to the user by reading some text to the user, by 
printing some information, and so forth. The second de- 
vice then waits for some user input (box 98) which is 
then sent back to the controlled device (box 99). A feed- 
back received from the controlled device may be pro- 
vided to the user (not illustrated in Figure 9). The second 
device may either wait for another user input (arrow 
100), or it may return to a state where is expects some 
user interface information (arrow 101), or it returns to 
the initial state (arrow 102). 

[0088] There is an other scheme conceivable where 
the computer device with limited user-interface (control- 
led device) initiates the whole process, in this case, the 
controlled device sends user interface information to a 
particular second device. If there is a plurality of devices 
within reach of the controlled device, then the to-be con- 
trolled device or the user may chose one. Before user- 
interface information is sent out, the device or user might 
want to check whether there is another device in vicinity 
which has the right user-interface. This can be done by 
simply looking at the information stored in a list with 
services, if there is no such list maintained, the control- 
led device might simply decide to transmit the user-in- 
terface information hoping that there is in fact a device 
in vicinity which is able to receive and interpret the user- 
interface information. The second device receives the 
user interface information and provides a corresponding 
user interface to the user. The user then uses this user 
interface to input information. The user's input is then 
sent to the controlled device where the input is handled 
and/or executed. In an optional step some feedback is 
returned to the second device to indicate to the user that 
the controlled device handled or executed his com- 
mand. 

[0089] An extension of the above described scheme 
is addressed below (see Figure 6). This extension pro- 
vides for a split of user-interface source (device 67) and 
the command target 60, as illustrated in Figure 6. The 
controlled device 60 does not necessarily have to supply 
the entire user-interface description (which could be- 
come quite large - e.g. when lots of graphical elements 
are deployed) but only parts of it. In this case the device 
60 just sends a partial user-interface 63 (e.g. a text-only 
version) via the wireless communications path 66 to the 
controller 62. Or the controlled device 60 might supply 
only an initial user-interface description 63 (e.g. a URL) 
or "pointer". 

[0090] The actual user-interface 65 or the missing 
part(s) (e.g. graphics 69) are then fetched from other 
locations (e.g. a file pre-installed on the controller or 
some WWW server on the Internet) and combined into 
a unified user-interface presentation 68 on the screen 
14 of the device 62. In the present example the actual 
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user-interface is fetched from a WWW server 67 using 
an http request 64. This would allow to keep the user- 
interface description 63 in the device 60 very small and 
would only require the implementation of a simple com- 
mand- and parameter parsing capability in the device 
60 itself. 

[0091] Another, more complex or (semi-)automatic 
user- interface implementation is now addressed: While 
the controlled device might only implement basic com- 
mands, using scripting techniques, these commands 
could be combined by the controller to form more pow- 
erful compound commands and also allowing to autom- 
atize certain repetitive task. Given e.g. a browser 
equipped with some flexible scripting language one 
could for example automatize task like the following: 

• Ask the user to insert a CD 

• Once a CD is inserted, request and display the cur- 
rent CD's identification code. 

• Search that CD's id-code in some Web-based da- 
tabase (e.g. "CDDB") 

• Copy title and author of that CD into the title-field of 
the CD-player's Ul (see example below) and submit 
that request. 

• Start over. 

[0092] Thus the user could simply insert one CD after 
the other into a CD-player (controlled device in the 
present example) while the PC (controller in the present 
example) searches and automatically programs the cor- 
responding CD-labels into the CD-player. 
[0093] The present invention can be improved by al- 
lowing the translation of user-interface controls. In this 
case, the controlling device acts as a format translator, 
i.e. converts user-interface elements to/from different 
formats or media. The controlling device could provide 
speech synthesis and "read" some text-message to a 
vis ion -impaired or occupied person (e.g. during car-driv- 
ing). Similarly, it could convert spoken commands to en- 
ter data into an input field or activate a control -element 
(a spoken "button" click). Such conversion is of course 
eased or encouraged by user-interface representation 
formats, that do not make any assumptions about the 
actual physical user-interface capabilities available in a 
device, but rather specify abstract functional levels of 
controls ; e.g. WML does not specify minimal display siz- 
es in pixels nor require a minimum number of fonts to 
be available for menus and text-output but rather spec- 
ifies "selection", "input" and "activation" capabilities. 
Menu -texts can either be displayed or e.g. read to the 
user and the user can type his/her answer or can simply 
speak to the device. 

[0094] If a manufacturer can rely on the fact that sel- 
dom usedf unctions need not be controlled via a device's 



front-panel but rather using a better suited external de- 
vice, the amount of user-interface code for complicated, 
seldom used functions can be drastically reduced, 
which yields much easier less error-prone software de- 

5 velopment and consequently shorter time-to-market 
and considerable price benefits due to quicker develop- 
ment. A controllable device in accordance with the 
present invention can be simpler and less expensive. 
[0095] Given a minimal communication range (say 1 5 

to - 20m) the device to be controlled need not even be in 
the same room or flat. It could be in the basement or on 
the roof (e.g. heating, air condition, antennas, cable tun- 
ers, satellite receivers, etc.). 

[0096] The standardized communication channel be- 
15 tween a controlled device and a controller can be ex- 
tended (this technique is usually known as "proxy") to 
allow the bridging of larger distances and to allow re- 
mote-control and remote-diagnosis capabilities. Exam- 
ple: building and heating control might require special 
20 knowledge such that even a good user-interface does 
not enable a customer to correctly adjust certain set- 
tings. By temporarily relaying the user-interface to a 
specialized firm, some external specialist may configure 
or diagnose a remote system. Sometimes such world- 
25 wide access to home equipment might be convenient 
for a "normal" user, too, because it will allow the user to 
control systems at home. 

[0097] It would also be possible to use HTML as ex- 
change format. This allows more flexible, more powerful 
30 user-interfaces but would be less elegant and compact 
than an WLM implementation. Any other markup lan- 
guage can be used as well. 

[0098] If the host-device (the controller) supports 
drag-and-drop capabilities this functionality can be ex- 

35 pjoited - e.g. to copy an appointment from a PDA's agen- 
da to the downloaded user-interface of a wrist-watch. 
This drag and drop can work between host-applications 
and downloaded user-interfaces. There can also be a 
drag and drop between devices. If more than one device 

40 j S currently controlled by the same controller, that host 
can act as a mediator, i.e. one could drag and drop in- 
formation between two controlled devices (e.g. copy a 
phone number stored in a wrist-watch to a mobile 
phone). 

45 [0099] Many of the benefits of the present invention 
become obvious when reading the specification. The 
present scheme allows to make use of larger, better 
readable displays (e.g., a larger color graphics display), 
better/faster input capabilities (e.g.,a full-fledged key- 

50 board, or pointing device), better suited I/O interfaces 
(e.g. a printer or audio system). It is certainly easier to 
program a mobile phone or a wrist-watch using a PDA 
or a Computer screen and a keyboard than keying-in 
data on a tiny numeric keyboard. According to the 

55 present invention, the user can use a mouse, pen, or 
any other pointing device provided by the device to con- 
trol features of a device which has no such mouse, pen, 
or other pointing device. 



12 



23 



EP1 069 500 A1 



24 



[0100] In the drawings and specification there has 
been set forth a preferred embodiment of the invention 
and, although specific terms are used, the description 
thus given uses terminology in a generic and descriptive 
sense only and not for purposes of limitation. s 
[0101] The present invention can be in part or as a 
whole be implemented by or on a special computer de- 
vice or a general purpose computer device. This can be 
done by implementing the invention in some form of 
computer program. Computer program in the present 10 
context means an expression, in any language, code or 
notation, of a set of instructions intended to cause a 
computer device to perform a particular function either 
directly or after either or both of the following: 

15 

a) conversion to another language, code or nota- 
tion; 

b) reproduction in a different material form. 

20 

[0102] It will be appreciated by persons skilled in the 
art that the present invention is not limited by what has 
been particularly shown and described herein above. 
Rather the scope of the invention is defined only by the 
claims which follow: 25 



Claims 

1. Method for controlling a first computer device with 30 
limited user-interface via a remote second compu- 
ter device, whereby both computer devices are in- 
terconnected through a wireless communication 
channel and both computer devices support a com- 
mon communications protocol, comprising the 35 
steps: 

• sending user-interface information from the 
first computer device to the second computer 
device, 40 

• providing a user-interface by the second com- 
puter device using the user- interface informa- 
tion, 

45 

• receiving user input at the second computer de- 
vice, 

• sending the user input to the first computer de- 
vice, and 50 

• executing the user input at the first computer 
device. 

2. The method of claim 1 , wherein a user-interface de- 55 
scription is used to transmit the user-interface infor- 
mation. 



3. The method of claim 1 , wherein the second compu- 
ter device announces its services to the first com- 
puter device prior to the sending of user-interface 
information from said first computer device to said 
second computer device. 

4. The method of claim 1 , wherein the wireless com- 
munication channel is automatically established be- 
tween the first computer device and the second 
computer device. 

5. The method of claim 1 . wherein the second compu- 
ter device comprises a display for providing a user- 
interface by displaying it on the display. 

6. The method of claim 1 , wherein the second compu- 
ter device comprises a keyboard for receiving the 
user input. 

7. The method of claim 1 , wherein a markup language 
is used for sending user-interface information from 
the first computer device to the second computer 
device. 

8. The method of claim 7, wherein WML is used as the 
markup language. 

9. The method of claim 7, wherein the second compu- 
ter device comprises a browser software which pro- 
vides the user-interface using the user-interface in- 
formation. 

10. The method of claim 1, wherein a special protocol 
is used for sending the user input to the first com- 
puter device. 

11. The method of claim 11, wherein the hypertext 
transport protocol, or the wireless session protocol 
is used as the special protocol. 

12. The method of claim 1 comprising the step of send- 
ing a feedback from the first computer device to the 
second computer device. 

13. The method of claim 12, wherein the feedback in- 
dicates whether the execution of the user input at 
the first computer device was successful. 

14. The method of claim 1, wherein the first computer 
device initiates the process by sending user-inter- 
face information to the second computer device. 

15. The method of claim 1 , wherein the second compu- 
ter device requests the first computer device to 
send the user-interface information. 

16. A system with 



25 



EP 1 069 500 A 1 



26 



a first computer device (70) comprising a limit- 
ed user-interface, a first processor (77), a first 
transceiver (73, 74), a first memory (76), and a 
user-interface manager (71); 

a second computer device (700) comprising a 
second user- interface (705), a second proces- 
sor (704), a second transceiver (730, 740) : and 
a second memory (703); and 

a wireless communications channel (81 , 82) for 
communication between the first computer de- 
vice (70) and the second computer device 
(700); 



10 



15 



whereby the user-interface manager (71) con- 
trols the sending of user-interface information 
via the first transceiver (73, 74), the wireless 
communications channel (81 , 82) and the sec- 
ond transceiver (730, 740) to the second com- 20 
puter device (700), 



21. The system of claim 16, whereby the second com- 
puter device (700) comprises means that allow a 
user to initiate a process where the second compu- 
ter device (700) requests the user-interface infor- 
mation from the first device (70). 

22. The system of claim- 16, further comprising a third 
computer device (67) which has a third processor, 
a third transceiver, and a third memory storing part 
of the user-interface information. 

23. The system of claim 22, whereby a first part of the 
user-interface information is transmitted by the first 
computer device (70) and a second part of the user- 
interface information is transmitted by the third com- 
puter device (67). 

24. The system of claim 23, whereby the first part of the 
user-interface information is a pointer to a portion 
of the third memory where the second part of the 
user-interface information is stored. 



whereby the second controller (710) provides 
a user-interface on the second user-interface 
(705, 750) using the user-interface information, 

whereby the second computer device (700) re- 
ceives user input through the second user-in- 
terface (705, 750), 

whereby the second computer device (700) 
sends the user input via the second transceiver 
(730, 740), the wireless communications chan- 
nel (81 , 82), and the first transceiver (73, 74) to 
the first computer device (70), and 

whereby the first computer device (70) exe- 
cutes the user input. 

1 7. The system of claim 1 6, whereby the first transceiv- 
er (73 : 74) and second transceiver (730, 740) auto- 
matically establish the wireless communication 
channel (81, 82) between the first computer device 
(70) and the second computer device (700). 

18. The system of claim 16, whereby the second user- 
interface comprises a display for displaying a user- 
interface. 

19. The system of claim 16, whereby the second user- 
interface (705, 750) comprises a keyboard for re- 
ceiving the user input. 

20. The system of claim 16, whereby the second com- 
puter device (700) comprises a browser software 
for providing the user-interface on the second user- 
interface (705, 750) using the user-interface infor- 
mation. 



25. A computer program product comprising a compu- 
ter readable medium, having thereon: 

25 computer program code means, when said program 
is loaded, to make a computer device which com- 
prises a limited user-interface, a processor, a trans- 
ceiver for interfacing through a wireless communi- 
cations channel with a remote computer device, a 

30 memory, and a user-interface manager execute 
procedure to: 



send user-interface information through the 
wireless communications channel to the re- 
mote computer device, 



35 



40 



45 



receive a user input that a user generated at 
the remote computer device via the wireless 
communications channel, 

execute the user input, 

send feedback through the wireless communi- 
cations channel to the remote computer device. 



26. A computer program product comprising a compu- 
ter readable medium, having thereon: 
computer program code means, when said program 
is loaded, to make a computer device which com- 
50 prises a user-interface, a processor, a memory, a 
user- interface, and a transceiver for interfacing 
through a wireless communications channel with a 
remote user-interface limited computer device, ex- 
ecute procedure to: 

55 

• receive user-interface information from the us- 
er-interface limited computer device through 
the wireless communications channel, 
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provide a user-interface using the received us- 
er-interface information, 

receive user input, 

5 

send the user input via the wireless communi- 
cations channel to the user-interface limited 
computer device, 

receive feedback through the wireless commu- 10 
nications channel from the user-interface limit- 
ed computer device, 

provide the feedback to a user. 

15 



20 
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<?xml version="l . 0 " ?> 

<!DOCTYPE WML SYSTEM "wml.dtd"> 
<WML> 

<TEMPLATE> 

<DO TYPE= " HELP " LABEL= " help " > <GO URL="#help"/> </DO> 
</TEMPLATE> 

<CARD NAME= "General settings" NEWCONTEXT- * TRUE " > 
<BR ALIGN= "CENTER" /> 
Welcome to 
<BR ALIGN= "CENTER" /> 

<BIGxB>SONY CD-Player 990X</Bx/BIG> 
<BR/> <BR/> <BR/> 

Please select the function you want to configure ; <BR/ 
<A TITLE="set/edit CD labels " >CD label s<GO 
URL= " #CD_names " /></A> 

<BR/> 

<A TITLE= " select play mode " >Play mode<GO 
URL= " #pl ay_mode ■ /></A> 

</CARD> 

<CARD NAME= " CD_names " > 

<BIGxB>CD names : </Bx/BIG> 
<BR/> 

Enter a name for the currently inserted CD: 
<BR/> 

< INPUT TYPE=" TEXT" KEY= " CD__NAME " FORMAT- " MMMMMM " 

EMPTYOK= " TRUE " / > 
<DO TYPE= " ACCEPT " LABEL="ok"> 
<GO 

URL="bluetooth: / /sony_cdp_990/cd__titles /add; $ <CD_NAME) "/> 
</DO> 
</CARD> 

<CARD NAME="play__mode"> 

<BIG><B>Play-modes : </Bx/BIG> 
<BR/> 

Select one of the following play modes: 
<BR/> 

<SELECT TITLE= "play-modes : " KEY= " PLAYMODE " 
DEFAULT^ "Normal " > 
<OPTION VALUE=" Normal" TITLE= "Normal • /> 
<OPTION VALUE=" Random" T I TLE= " Random" /> 
<OPTION VALUE=" Shuffle" TITLE= "Shuffle" /> 
<OPTION VALUE^UserDef " TITLE= "User-Defined" /> 

</SELECT> 

<DO TYPE= "ACCEPT" LABEL="ok"> 
<GO 

URL="bluetooth: / /sony_cdp_990/playmode/select ; $ (PLAYMODE) "7 
</DO> 
</CARD> 

<CARD NAME^"help"> 

Some help text here. 
</CARD> 

</wml> FIG. 10 
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